Ollama (Orchestrator / Inference Engine)

Ollama is a model runner (also called an inference engine). Ollama runs the LLM Models like: Llama, Gemma, Phi, qwen etc

[ Go Client ] 
       │  (HTTP / JSON REST API)
       ▼
[ Ollama Binary (Executable) ] ───(Loads into memory)───► [ Model Weights (Static Data File) ]
       │
       ▼  (Performs Matrix Multiplication Math)
[ CPU / GPU Hardware Compute ]
      

Can Ollama run any LLM model?

Ollama understands GGUF format. Any model which can be converted to GGUF format.
1. Direct Support: Dozens of models (Llama 3, Gemma 2, Mistral, Qwen, DeepSeek). Download and run
2. Custom Models: Convert to GGUF format and run.

Install, Run Agent in Ollama

///////// Install /////////
$ curl -fsSL https://ollama.com/install.sh | sh           // Linux
// Windows. Download from website and run installer

////////// Run Ollama /////////
// Linux. olllama is in /etc/systemd/system/ollama.service
$ sudo systemctl restart | start | status ollama
// Windows. Start Ollama from Start Menu or Desktop Icon

/////// Download Model /////////
// Linux, Windows
$ ollama run qwen2.5:1.5b

/////// Run Model ////////////
$ ollama list
$ ollama run llama3.2
Enter /bye in ollama prompt

//////// Check Model is running /////////
// Linux, Windows
$ ollama ps
$ curl http://localhost:11434/api/generate -d '{
  "model": "qwen2.5:1.5b",
  "prompt": "Hi",
  "stream": false
}'
      

Hit AI Model (qwen's) REST APIs

Input: "indarpur unfollowed for 2 weaks" (with a typo on 'weeks').
Output: The model interpreted the command ("Return JSON only."), extracted the semantic meaning of misspelled input phrase, corrected the spelling error, mapped it to key-value variables, and wrapped it cleanly in a markdown code block.

{
  "status": "Unfollowed",
  "duration": "2 weeks"
}
        

$ cat test.json
{
  "model":"qwen2.5:1.5b",
  "messages":[
    {
      "role":"system",
      "content":"Return JSON only."
    },
    {
      "role":"user",
      "content":"indarpur unfollowed for 2 weaks"
    }
  ],
  "temperature":0,
  "stream":false
}

$ curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d @test.json
{"id":"chatcmpl-421","object":"chat.completion","created":1781543972,"model":"qwen2.5:1.5b","system_fingerprint":"fp_ollama","choices":[{"index":0,"message":{"role":"assistant","content":"```json\n{\n  \"status\": \"Unfollowed\",\n  \"duration\": \"2 weeks\"\n}\n```"},"finish_reason":"stop"}],"usage":{"prompt_tokens":28,"completion_tokens":24,"total_tokens":52}}
      

Qwen Logs

//Since Ollama is running inside a systemd service unit, logs are managed natively by journald.

$ journalctl -u ollama.service -f
Jun 15 22:49:31 Amit ollama[134899]: slot update_slots: id  0 | task 22 | cached n_tokens = 3, memory_seq_rm [3, end)
Jun 15 22:49:31 Amit ollama[134899]: slot init_sampler: id  0 | task 22 | init sampler, took 0.01 ms, tokens: text = 28, total = 28
Jun 15 22:49:31 Amit ollama[134899]: srv  log_server_r: done request: POST /completion 127.0.0.1 200
Jun 15 22:49:32 Amit ollama[134899]: slot print_timing: id  0 | task 22 | prompt eval time =     405.33 ms /    25 tokens (   16.21 ms per token,    61.68 tokens per second)
Jun 15 22:49:32 Amit ollama[134899]: slot print_timing: id  0 | task 22 |        eval time =     786.05 ms /    24 tokens (   32.75 ms per token,    30.53 tokens per second)
Jun 15 22:49:32 Amit ollama[134899]: slot print_timing: id  0 | task 22 |       total time =    1191.39 ms /    49 tokens
Jun 15 22:49:32 Amit ollama[134899]: slot print_timing: id  0 | task 22 |    graphs reused =         41
Jun 15 22:49:32 Amit ollama[134899]: slot      release: id  0 | task 22 | stop processing: n_tokens = 51, truncated = 0
Jun 15 22:49:32 Amit ollama[134899]: srv  update_slots: all slots are idle
Jun 15 22:49:32 Amit ollama[134899]: [GIN] 2026/06/15 - 22:49:32 | 200 |  1.409126234s |       127.0.0.1 | POST     "/v1/chat/completions"

// Increase logs level
$ sudo vim /etc/systemd/system/ollama.service
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_DEBUG=1"

$ sudo systemctl daemon-reload
$ sudo systemctl restart ollama
      

Use Cases

CRM Voice Search feature (Speak donot Write)

Business Problem?

You have a frontend CRM app written in react native, which hits REST endpoints on golang backend and pushes and gets data. App has ViewAllRequirements page which shows all requirements in row format. When user clicks on 1 requirement it shows complete information about the requirement. Now we want a feature on frontend app(smart search), that employee or owner SPEAK on Frontend app example:

All requirements which are not followed up since 2 weeks OR
All requirements from Ajnara grace OR
(in Hindi) Mujhe piche 5 dino k untouched requirement do OR
indarpur unfollowed for 2 weaks   //Spelling mistake. Area name mistake
        

Frontend should show the authorized data. What changes are required on frontend and what are required on backend? We want 0 cost solution, backend would hosted on AWS.

Solution

Uninstall Ollama and Qwen

$ sudo systemctl status ollama
[sudo] password for amit:
● ollama.service - Ollama Service
     Loaded: loaded (/etc/systemd/system/ollama.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2026-06-21 10:41:11 IST; 5min ago

$ ollama list
NAME            ID              SIZE      MODIFIED
qwen2.5:1.5b    65ec06548149    986 MB    5 days ago

$ ollama rm qwen2.5:1.5b
 ollama list
NAME    ID    SIZE    MODIFIED

sudo rm -rf /usr/share/ollama/.ollama/models
sudo systemctl stop ollama
sudo systemctl disable ollama
sudo rm /etc/systemd/system/ollama.service
sudo systemctl daemon-reload

sudo rm -f /usr/local/bin/ollama
sudo rm -f /usr/bin/ollama
sudo rm -rf /usr/share/ollama
sudo userdel ollama
sudo groupdel ollama